Monitor method and monitor system thereof wherein mask is used to cover image for detecting object

ABSTRACT

A monitor method for detecting an object includes capturing an image; calculating an initial probability of the object existing in the image; applying a mask to cover a first portion of the image if the initial probability is higher than a threshold; calculating a first probability of the object existing in the image excluding the first portion; and using at least the first probability to detect the location the object in the image.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The disclosure is related to a monitor method and a monitor devicethereof, and more particularly, a monitor method and a monitor devicethereof where a mask is used to cover an image for detecting an object.

2. Description of the Prior Art

With the increasing demand for security surveillance, the demand foranalyzing monitor images has also increased. For example, a surveillancecamera operated 24 hours a day can generate videos 8760 hours a year,and there may be hundreds of surveillance cameras in a building. Hence,the human resource for monitoring and even analyzing the videos andimages can be overwhelming. Presently, a lot of manpower is used tomonitor the screens or images for assuring the security. However, thiswill lead to higher cost and human error because the attention ofsecurity practitioners is limited. In the field, an automatic solutiontaking reasonable resource for analyzing the images captured by camerasis in shortage.

SUMMARY OF THE INVENTION

An embodiment provides a monitor method for detecting an objectincluding capturing an image; calculating an initial probability of theobject existing in the image; applying a mask to cover a first portionof the image if the initial probability is higher than a threshold;calculating a first probability of the object existing in the imageexcluding the first portion; and using at least the first probability todetect the location the object in the image.

Another embodiment provides a monitor system for detecting an object,including a surveillance camera configured to capture an image; and aprocessor configured to apply a mask to cover a first portion of theimage, calculate a first probability of the object existing in the imageexcluding the first portion, and use at least the first probability todetect the object in the image.

These and other objectives of the present invention will no doubt becomeobvious to those of ordinary skill in the art after reading thefollowing detailed description of the preferred embodiment that isillustrated in the various figures and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an image including an object to be detected.

FIG. 2 is a flowchart of a monitor method according to an embodiment.

FIG. 3 illustrates a processed image obtained by processing the image ofFIG. 1 by means of the monitor method of FIG. 2.

FIG. 4 illustrates an example where a portion and a subsequent portioncovered by the mask partially overlap with one another.

FIG. 5 illustrates an example where the dimension of the mask isadjusted.

FIG. 6 to FIG. 9 illustrate an example where the mask respectivelycovers the portions of the image.

FIG. 10 to FIG. 13 illustrate an example where the mask is moved tosequentially cover sub-portions in the selected portion of FIG. 6 toFIG. 9.

FIG. 14 illustrates the processed image generated by processing theimage shown in FIG. 6 to FIG. 13.

FIG. 15 illustrates a monitor system for detecting an object accordingto embodiments.

DETAILED DESCRIPTION

In order to deal with the problem mentioned above, a monitor method anda monitor device can be provided according to embodiments to analyze theimages of a monitored space.

FIG. 1 illustrates an image 100 including an object OBJ to be detected.FIG. 2 is a flowchart of a monitor method 200 according to anembodiment. FIG. 3 illustrates a processed image 110 obtained byprocessing the image 100 of FIG. 1 by means of the monitor method 200.FIG. 1 and FIG. 3 are merely of an example instead of limiting the scopeof embodiments. By means of the monitor method 200, the image 100 can beprocessed to generate a processed image 110.

The object OBJ can be a person or a predetermined object. For example,the image 100 can be an image of an office, and the purpose of securitysurveillance can be monitoring whether any person exists at an abnormaltime (e.g., weekend or midnight) or whether any person abnormallyappears in a restricted area (e.g., an area close to a vault). In orderto detect the existence and/or the location of the object OBJ, themonitor method 200 can be used. The monitor method 200 can include thefollowing steps.

Step 202: capture the image 100;

Step 204: calculate an initial probability PO of the object OBJ existingin the image 100;

Step 206: determine whether the initial probability PO is higher than athreshold; if so, enter Step 210; else enter Step 208;

Step 208: determine the object OBJ does not exist in the image 100;enter Step 202;

Step 210: determine the object OBJ exists in the image 100;

Step 215: apply a mask M to cover an ith portion Ai of the image 100;

Step 220: calculate an ith probability Pi of the object OBJ existing inthe image 100 excluding the ith portion Ai and remark the ithprobability Pi at the portion Ai;

Step 225: determine whether the location of the mask M is at thepredetermined location; if so, enter Step 235; else enter Step 230;

Step 230: add 1 to i; enter Step 215; and

Step 235: determine the location of the object OBJ.

In Step 202 to Step 210, an initial classification can be performed todetermine whether the object OBJ (e.g., a person or a specific object)exists. In Step 204 and Step 206, a machine learning model such as aconvolutional neural network (CNN) model can be used, and thedetermination made by the model can be more and more accurate bytraining to adjust the weights used for calculation.

The threshold mentioned in Step 206 may be, for example, 90% or anappropriate threshold obtained by experiment and calculation such as theresult of machine learning. Only if the object OBJ is determined toexist, Step 215 to Step 235 can be performed to determine the locationof the object OBJ.

In Step 215 to Step 235, the mask M can be moved to cover differentportions of the image 100 to obtain the probabilities corresponding todifferent portions, and the probabilities can be remarked at differentportions of the image 110. The location of the object OBJ can beaccordingly determined.

In Step 215 to Step 235, the mask M can cover an ith portion Ai of theimage 100. Here, the variable i can be an integer larger than zero. Themask M can have a predetermined dimension such as m pixel(s)×n pixel(s),where m and n can be integers larger than zero. When the ith mask isapplied to cover the portion Ai, there can be one of the three scenariossaid below.

(Scenario-i)

If the mask M fails to cover any part of the object OBJ in the image100, the ith probability Pi of the object OBJ existing in the image 100excluding the ith portion Ai in Step 220 can be as high as the initialprobability PO said in Step 204.

(Scenario-ii)

If the mask M covers a part of the object OBJ in the image 100, the ithprobability Pi of the object OBJ existing in the image 100 excluding theith portion Ai in Step 220 can be a value lower than the initialprobability PO said in Step 204.

(Scenario-iii)

If the mask M covers all of the object OBJ in the image 100, the ithprobability Pi of the object OBJ existing in the image 100 excluding theith portion Ai in Step 220 can be a lowest value (for example, zero).

When a plurality of portions (e.g., A1, A2, A3 . . . ) of the image 100are covered in sequence, the obtained probabilities (e.g., P1, P2, P3 .. . ) can be respectively remarked to the portions to generate theprocessed image 110, and the location of the object OBJ can bedetermined by means of the processed image 110. Below, FIG. 1 and FIG. 3can be an example to describe the method 200.

In FIG. 1, the mask M can be shifted to sequentially cover the portionsA1, A2, A3 . . . A24 to obtain the probabilities P1, P2, P3 . . . P24.By remarking the probabilities P1, P2, P3 . . . P24 to the portions A1,A2, A3 . . . A24, the processed image 110 of FIG. 3 can be obtained. Inthe example of FIG. 3, the probabilities P3, P4, P9 and P10 (remarked atthe portions A3, A4, A9 and A10 of the image 100) are relatively lowerthan other probabilities because when the mask M covers the portions A3,A4, A9 and A10, a part of the object OBJ is covered as the scenario-iimentioned above. Hence, in Step 235, the location of the object OBJ canbe determined to be at the portions A3, A4, A9 and A10 of the image 100by reviewing all probabilities in FIG. 3. The arrows in FIG. 1 are usedto indicate the shift of the mask M.

Regarding Step 225, if the location of the mask M is NOT at thepredetermined location, the mask M can be moved to cover a subsequentportion of the image 100, and the location of the covered portion of theimage 100 can be adjusted regularly each time. The predeterminedlocation said in Step 225 can be a predefined final location. If thelocation of the mask M is at the predetermined location, the mask M maynot be moved to cover a subsequent portion of the image 100.

For example, in FIG. 1 and FIG. 3, the portion A1 being firstly coveredby the mask M can be at the upper left corner of the image 100. Then,the mask M can be moved to the right to cover the portion A2. So on andso forth, the mask M can be move to the right to sequentially cover theportions A3, A4, A5 and A6. Since the portion A6 is at the upper rightcorner of the image 100, after covering the portion A6 to obtain theprobability P6, the mask can be moved to cover the portion A7 below theportion A1. Likewise, the mask M can be continuously shifted tocalculate the probabilities corresponding to different portions of theimage 100 until the mask M is moved to the predetermined location saidin Step 225. For example, in FIG. 1 and FIG. 3, the predeterminedlocation said in Step 225 can be at the portion A24 at the lower rightcorner of the image 100.

In FIG. 1 and FIG. 3, a portion and a subsequent portion covered by themask M are adjacent with one another and not overlapped with oneanother; however, this is merely an example instead of limiting thescope of embodiments. According to other embodiments, a portion and asubsequent portion covered by the mask M can partially overlap with oneanother as shown in FIG. 4.

For example, if the width of the image 100 said in Step 202 includes 192pixels, the width of the mask M is 24 pixels, and a portion (e.g. A1)and a subsequent portion (e.g. A2) being sequentially masked have anoverlap width of 23 pixels, the mask M can be horizontally moved for 168(i.e. 192-24) times from a leftmost portion (e.g. A1) to a rightmostportion (e.g. A169) of the image 100 to cover 169 portions forcalculating 169 probabilities of the object OBJ existing in the uppersection of image 100. In other words, the mask M can be moved by 1 pixeleach time. By overlapping a portion with a subsequent portion covered bythe mask M, the resolution of detecting the object OBJ can beeffectively increased; however, the amount of calculation is alsoincreased, and more computing resources are needed.

According to embodiments, the dimension of the mask M can be adjusted sothat the dimension of the portion can be different from the dimension ofthe subsequent portion covered by the mask M. For example, as shown inFIG. 5, the dimension of the portion A2 can be different from thedimension of the preceding portion A1, where the dimension of mask M isshrunk after being moved from the portion A1 to the portion A2. FIG. 5is merely an example, according to requirements, the mask M can beenlarged for calculating the probility.

According to embodiments, the mask M can cover a first portion of theimage 100, and cover a second portion of the image 100 afterward whenStep 215 is performed for another time, where the first portion can beinside the second portion. In this manner, the resolution of detectingthe object OBJ can hence be further improved. An example is describedbelow.

FIG. 6 to FIG. 9 illustrate the mask M respectively covers the portionsA1 to A4 of the image 100 in an example. In the example, the image 100can be a picture of an office, and the objected OBJ to be detected canbe a person. The portions A1, A2, A3 and A4 can be at the upper leftportion, upper right portion, lower left portion, and lower rightportion of the image 100. In this example, the portions A1 to A4 do notoverlap with one another. As mentioned in Step 215 and Step 220, fourprobabilities P1, P2, P3 and P4 of the object OBJ existing in the image100 excluding the covered portions can be calculated, and theprobabilities P1, P2, P3 and P4 can be respectively remarked to theportions A1 to A4.

Regarding FIG. 6 to FIG. 9, since the mask M covers the object OBJ mostwhen the mask M covers the portion A4 as shown in FIG. 9, theprobability P4 can be lower than the probabilities P1 to P3. The portionA4 can thus be selected, and it can be determined that the location ofobject OBJ is more related to the portion A4. For example, the locationof object OBJ is in the portion A4, or the ratio of the object OBJ inthe portion A4 is more than that in each of the portions A1 to A3.

If it is sufficient to determine that the object OBJ is more related tothe portion A4, the process can be stopped. However, if a higherresolution is pursued, the portion A4 can be partially covered tofurther analyze the location of the object OBJ. FIG. 10 to FIG. 13illustrate the mask M being moved to sequentially cover sub-portionsA41, A42, A43 and A44 in the selected portion A4 to more preciselydetect the location of the object OBJ.

The operation related to FIG. 10 to FIG. 13 can be similar to theoperation of FIG. 6 to FIG. 9. The portion A4 can be regarded an imageto be analyzed, and the four sub-portions A41, A42, A43 and A44 can becovered in sequence to respectively calculate the probabilities P41,P42, P43 and P44. The probabilities P41, P42, P43 and P44 can berespectively remarked to the sub-portions A41, A42, A43 and A44 tofurther detect the location of the object OBJ being more related towhich sub-portion(s) in the selected portion (e.g. A4). Hence, theresolution of detecting the object OBJ can be improved. In the exampleof FIG. 10 to FIG. 13, the sub-portion A41 can be selected since theprobability P41 is relatively low, and the location of the object OBJcan be more related to the sub-portion A41.

By means of selecting at least one portion (e.g., A4) and furthercovering sub-portions of the selected portion to detect the location ofthe object OBJ, the computing resources can be saved. For example,regarding FIG. 6 to FIG. 9, since the location of the object OBJ is morerelated to the portion A4, the portions A1 to A3 can be optionallyomitted without being further analyzed. Hence, the operation of theexample shown in FIG. 6 to FIG. 13 can save more resources than that ofFIG. 1.

The monitor method provided by embodiments can also reduce the use ofweights of a neural network model. In FIG. 1 and FIG. 3 to FIG. 13, themask M is used to cover a portion or a sub-portion of the image 100, anda neural network model can be used to analyze the covered image tocalculate probability of the object OBJ existing in the image 100. Thelocation and/or the dimension of the mask M can be adjusted; however,the neural network model (e.g., CNN model) used for calculation andclassification in each stage can be similar or even the same. In otherwords, the same set of weights used for calculating differentprobabilities. Hence, the computing resources can be further reduced.

When the mask M is of a predetermined dimension corresponding to apredetermined resolution, a sub-portion may not be partially covered toanalyze the location of the object OBJ. If a higher resolution isneeded, as the operation shown in FIG. 10 to FIG. 13, a selectedsub-portion can be further partially covered by the mask M (with afurther reduced dimension) to calculate corresponding probabilities ofthe object OBJ existing in order to further analyze the location of theobject OBJ in the selected sub-portion.

The operations described in FIG. 1, FIG. 4, FIG. 5 to FIG. 13 can beoptionally applied in combination to analyze an image. For example, theoperations described in FIG. 1, FIG. 4 and FIG. 5 can be used to analyzean image to detect an object.

FIG. 14 illustrates the processed image 110 generated by processing theimage 100 shown in FIG. 6 to FIG. 13. The unit used on the vertical andhorizontal axes of FIG. 14 can be pixel or unit of length. The densityshown in FIG. 14 can be corresponding to the probability of the objectOBJ existing in the image 100, and the probability can be normalized ifrequired. As described in FIG. 1 to FIG. 13, the mask M can slide tocover a plurality of portions (and sub-portions) to calculate aplurality of probabilities, and the probabilities can be remarked in theprocessed image 110.

As shown in FIG. 14, the location of the objected OBJ can be observed asthe pattern OBJd; though the boundary of the pattern OBJd may not bevery clear, it may be sufficient for the purpose of securitysurveillance. If a higher resolution is needed, the mask M mentionedabove can be further shrunk in size and shifted by a smaller distanceeach time to more accurately locate the object OBJ. The pattern OBJd canbe displayed for a user to watch, or not be displayed to be merelyrecognized by a machine.

FIG. 15 illustrates a monitor system 1500 for detecting an object OBJaccording to embodiments. The monitor system 1500 can include asurveillance camera 1510, a warning device 1530, a display 1540 and aprocessor 1520 coupled to the surveillance camera 1510, the warningdevice 1530 and the display 1540. The surveillance camera 1510 cancapture the image 100 mentioned above. The surveillance camera 1510 canbe an optical camera, an infrared camera and/or a network InternetProtocol camera. The processor 1520 can used to perform the method 200and related operations, determinations and classifications mentionedabove. The display 1540 can selectively display the image 100, theprocessed image 120, the mask M in each stage, the data, the processand/or the result related to detection and analysis of the location ofthe object OBJ. The warning device 1530 can send a warning signal whenthe object OBJ is detected and/or the object OBJ is detected to be in apredetermined area. The warning signal can include warning sound,warning light or a notification to the police or the security guard.

In summary, by means of the monitor method and the monitor systemprovided by embodiments, an automatic solution can be provided to detectthe existence and the location of an object. The correctness can beassured, the resolution of detection can be flexibly adjusted and thecomputing resource can be saved. Hence, the long-standing problem in thefield can be effectively dealt with.

Those skilled in the art will readily observe that numerousmodifications and alterations of the device and method may be made whileretaining the teachings of the invention. Accordingly, the abovedisclosure should be construed as limited only by the metes and boundsof the appended claims.

What is claimed is:
 1. A monitor method for detecting an object,comprising: capturing an image; calculating an initial probability ofthe object existing in the image; applying a mask to cover a firstportion of the image if the initial probability is higher than athreshold; calculating a first probability of the object existing in theimage excluding the first portion; and using at least the firstprobability to detect the location the object in the image.
 2. Themonitor method of claim 1, further comprising: determining the objectdoes not exist in the image if the initial probability is lower than thethreshold.
 3. The monitor method of claim 1, further comprising:applying the mask to cover a second portion of the image; calculating asecond probability of the object existing in the image excluding thesecond portion; and using at least the first probability and the secondprobability to detect a location of the object in the image.
 4. Themonitor method of claim 3, wherein the first portion partially overlapswith the second portion.
 5. The monitor method of claim 3, wherein thefirst portion does not overlap with the second portion.
 6. The monitormethod of claim 3, wherein a dimension of the mask is adjusted so that adimension of the first portion is different from a dimension of thesecond portion.
 7. The monitor method of claim 3, wherein one of thefirst portion and the second portion is inside another one of the firstportion and the second portion.
 8. The monitor method of claim 3,wherein using at least the first probability and the second probabilityto detect the location of the object in the image, comprises: selectinga lower probability of the first probability and the second probabilitywherein the lower probability is corresponding to a selected portion ofthe first portion and the second portion; determining the location ofthe object is more related to the selected portion.
 9. The monitormethod of claim 3, wherein the first probability and the secondprobability are calculated using a same set of weights of a neuralnetwork model.
 10. The monitor method of claim 3, further comprising:selecting a selected portion from the first portion and the secondportion according to at least the first probability and the secondprobability; applying the mask to cover a first sub-portion of theselected portion wherein the mask is shrunk; and calculating a thirdprobability of the object existing in the image excluding the firstsub-portion; wherein the location of the object in the image is detectedaccording to at least the third probability.
 11. The monitor method ofclaim 10, further comprising: applying the mask to cover a secondsub-portion of the selected portion; and calculating a fourthprobability of the object existing in the image excluding the secondsub-portion; wherein the location of the object in the image is detectedaccording to at least the third probability and the fourth probability.12. The monitor method of claim 10, wherein the first probability andthe third probability are calculated using a same set of weights of aneural network model.
 13. The monitor method of claim 10, wherein thefirst sub-portion is not partially covered to analyze the location ofthe object when the mask is corresponding to a predetermined resolution.14. The monitor method of claim 3, further comprising: sending a warningwhen the object is detected and/or the object is detected to be in apredetermined area.
 15. The monitor method of claim 1, furthercomprising: determine whether a location of the mask is at apredetermined location; wherein the mask is not moved to cover asubsequent portion of the image when the mask is at the predeterminedlocation.
 16. A monitor system for detecting an object, comprising: asurveillance camera configured to capture an image; and a processorconfigured to apply a mask to cover a first portion of the image,calculate a first probability of the object existing in the imageexcluding the first portion, and use at least the first probability todetect the object in the image.
 17. The monitor system of claim 16,further comprising a display configured to display the image andselectively display the mask.
 18. The monitor system of claim 16,further comprising a warning device configured to send a warning whenthe object is detected and/or the object is detected to be in apredetermined area.
 19. The monitor system of claim 16, wherein theprocessor is further configured to determine the object exists if thefirst probability is higher than a threshold, apply the mask to cover asecond portion of the image, calculate a second probability of theobject existing in the image excluding the second portion, and use atleast the first probability and the second probability to detect alocation of the object in the image.
 20. The monitor system of claim 19,wherein the processor is further configured to use a same set of weightsin a neural network model for calculating the first probability and thesecond probability.